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BACKGROUND OF THE INVENTION 
Field of the Invention 

5 This invention relates to risk management in large-scale development projects, 

and more specifically to a web-based risk management tool and method for capturing, 
assessing, and prioritizing risks and implementing mitigation plans to more effectively 
manage risk. 

10 Description of the Related Art 

Due to the level of complexity of multi-million and even billion dollar programs 
such as the design, development and production of next generation weapon systems, 
aircraft or spacecraft, delivery of the product on time, within budget and with a high 
degree of quality assurance is a formidable task. 

15 A typical risk management process 10 incorporates a number of different tools to 

identify, assess, mitigate and track risk as illustrated in figure 1. In the initial phase, 
project management defines a Risk Management Plan detailing their approach for 
conducting risk management for the specific project based on customer requirements 12 
such as technical specifications, cost and schedule (step 14). The plan outlines how risks 

20 will be identified and assessed, how mitigation plans will be developed, how risks will be 
tracked and what tools and methods will be used. A program risk manager is tasked with 
implementing the plan and overseeing the risk management portion of the project. 

The first step is for each engineer to identify risks that may be associated with 
each of their tasks (step 16). To assist in this process, each engineer has a risk 

25 management tool such as Risk Radar on his or her own PC. The engineer may rely on 
his or her knowledge and experience, discussions with other engineers and a tool called 
Technical Risk Identification and Mitigation System (TRIMS), which is mainly used to 
identify process risks when a program is transitioning between phases such as 
development to production. The identified risks are then input into the risk management 

30 tool's local database on the PC. The engineer may be required to submit the identified 
risks to the program risk manager for consideration by the risk review board. 



To assess the risks (step 18), the engineer refers to a probability of occurrence 
table 20 of the type shown in figure 2 and based on his or her own knowledge and 
experience rates the probability of failure for a task occurring to be, for example, remote, 
unlikely, likely, highly likely or near certainty. These labels are then mapped to 

5 probabilities Pf and stored in the risk management tool. The selection of Pf is highly 
subjective based on the engineer's experience, interpretation of the task and 
interpretation of the labels. Next, the engineer refers to a severity of occurrence table 22 
of the type shown in figure 3 and selects the impacts 0, 1, 2, 3, 4, or 5 of failure on the 
basis of technical, schedule and cost impacts, which are also stored in the tool. The cost 

10 impact is specified as a percentage of the cost. Schedule impact is oftentimes specified 
as a percent slippage of a key milestone or, as shown in this case, in general terms of 
effect on key milestones. The tool calculates a risk factor Rf as the product of the 
probability of occurrence Pf and the largest impact. The risk factors are then prioritized 
as, for example, low, moderate and high based on thresholding rules set by the risk 

15 management plan and stored in the local database. The priority of the risk factors 
determines which risk receives special attention and resources and which will not and 
thus the accuracy and consistency with which they are determined is critical. In the 
current approach, the risk factors are highly subjective and dependent on the knowledge 
and expertise of the individual engineer. To accommodate any risk on any project, the Pf 

20 and Cf tables are very general. At this point, the engineer may be required to submit a 
report on the assessed risks to the program risk manager for consideration by the risk 
review board or may proceed directly to the development of risk mitigation plans. The 
risk manager maintains a master database of the aggregated reports from each engineer 
or team, monitors progress and reports back through risk review meetings on the status of 

25 risk mitigation. 

To develop a risk mitigation plan (step 24) y the engineer first refers to the risk 
assessment to determine the priority of the risk. A risk with a high risk assessment is 
expected to receive more resources for mitigation. In most cases, the engineer proposes a 
mitigation plan based on his or her knowledge and experience and submits the plan to the 

30 program risk manager for consideration by the risk review board. The board must 
consider all the program risk factors based on the risk factors and mitigation plans 



created independently by each engineer and, as constrained by available resources, make 
critical decisions on the mitigation plans to be funded and implemented. The board's 
decisions are only as good as the risk factors and plans provided. 

At this point, the identified risks, risk factors and accepted mitigation plans form 
5 an initial input to the integrated master plan that is the initial input to a program master 
schedule. A tool such as Microsoft Project is used to create a detailed schedule that links 
all the tasks and sub-tasks and mitigation plans together. As the project progresses, the 
engineers and managers continually update MS Project to track achieved milestones, 
slippage and reasons for revision. The progress information and trends from MS Project 
10 are forwarded to the risk manager to track and report risks (step 26) and analyze the 
information to determine whether new risks should be identified. 

Based on the current , status, the engineer updates the risk and mitigation plan 
status and submits the updates to the project risk manager (step 28). If risks are 
successfully mitigated and the milestones achieved, the risks are closed (step 30). If a 
1 5 risk changes, the engineer reassesses the probability of occurrence and impact (step 18), 
makes required adjustments to the mitigation plan and drafts a program plan adjustment 
report that is submitted to the project manager (step 32). In some cases, a Monte Carlo 
simulation such as Risk Plus is run on the schedule based on the most optimistic and 
pessimistic milestone dates and generates additional scheduling risks that should be 
20 considered. If a new risk or task appears, the engineer identifies the risk (step 16) and 
repeats the process. If the scope of the project changes, the process returns to the initial 
phase for reconsideration of the risk management plan (step 14). 

The standard risk management process both in determining an initial or baseline 
risk mitigation plan and in monitoring ongoing risks is highly subjective and based on 
25 the knowledge, experience and decision making capability of many engineers acting 
independently and in relative isolation to identify risks, determine the risk factors Rf and 
the development of mitigation plans. The Pf and Cf tables on which the engineers base 
their assessments are generalized to accommodate any program and any risk. 
Furthermore, the process is highly bureaucratic in that engineers must prepare and submit 
30 multiple reports with very similar information for consideration by the review board. 

US Patent 6,397,202 to Robert Higgens entitled "System and Method for 



Monitoring Risk in a System Development Program" has proposed a technique for an 
automated expert system to quantify various types of ongoing risk. The system defines a 
plurality of metrics such as requirements, staffing and source lines of code (SLOC) that 
relate to the successful completion of the project and projects a baseline for each metric. 

5 The system collects data for each metric over a period of time, compares the measured 
data to the baseline and assigns a risk based on the percentage the measured value 
deviates from the baseline. The metrics may be weighted to define the relative 
importance of the data measurements. When the project risk becomes sufficiently high, 
the program manager is alerted to the problem and can take measures to diminish the 

10 risk. 

Higgens relies on logical statements, algorithms, and the like, provided by experts 
who are knowledgeable in analyzing this type of data to determine an output risk level. 
The expert's rationale is, in effect, captured and stored with the rule-based system. 
Although a fully-automated expert system has obvious appeal, practice has shown that 
15 one size fits all systems that remove all human judgment are rarely effective, the risks 
encountered do not fit into predefined categories and are more complex than can be 
adequately handled by if-then type rules. Furthermore, Higgens assumes some baseline 
and then provides a system for monitoring the ongoing risks relative to that baseline and 
reacting to them. 

20 There remains an acute and present need to provide a risk management tool and 

method that is applicable to a wide range of projects but tailorable to a specific project, 
provides a better technique for assessing and mitigating risks to anticipate and minimize 
risks, and streamlines the risk management process. 

25 SUMMARY OF THE INVENTION 

The present invention provides a risk management tool and method for creating 
an improved initial risk management process that captures, assesses, and prioritizes risks 
and implements mitigation plans to more effectively identify, assess and manage risk in 
large-scale development projects. The invention utilizes a web-based system in which 

30 users and management can share and access risk information and can tailor the tool to a 
particular project to better anticipate and thus minimize the effects of risks. 



This is accomplished with a web-based tool that accesses a shared database and 
serves a plurality of users via a company intranet. The database stores accumulated 
information from past and current projects relating to risk. More specifically, 
information relating to specific risks that have been encountered by other users or on 

5 other programs is aggregated and periodically updated. A tailorable probability of 
occurrence table maps categories of risk such as assembly, engineering, lifetime, 
hardware, management, etc. to a probability of failure (Pf) based on standardized 
qualitative probability definitions. A tailorable severity of consequence table maps cost 
impact, schedule impact and technical/performance to a severity measure (Cf) based on 

10 standardized qualitative descriptions that are assigned program specific values. In 
particular, cost impact is sub-divided into development cost (NRE), unit cost (DTC) and 
operations and support cost (O/S) and specified in dollar amounts for the current project, 
and schedule impact is specified in actual days, weeks or months for the current project. 
Information is stored to augment a user's personal knowledge to select the most 

1 5 appropriate Pf and Cf entries. Mitigation plans for addressing prioritized risk factors are 
aggregated and periodically updated from a multitude of users and programs. 

The risk management process starts with the definition of the risk management 
plan based on customer requirements such as technical specifications, cost targets and 
schedule. The plan outlines how risks will be identified and assessed, how mitigation 

20 plans will be developed, how risks will be tracked and what tools will be used. In 
addition, the Pf and Cf tables are tailored to the current project. The Pf table is pruned to 
include only those categories that are relevant to the current project. The cost impact 
values in dollars for NRE, DTC and O/S and the schedule impact values in days, weeks 
or months are entered in the Cf table. This closely tailors risk assessment to a particular 

25 project and reduces subjectivity of users of the web-based tool. 

To create an initial set of risks, each user must first identify possible risks 
associated with his or her assigned task. To augment their own knowledge and 
experience, users will generate an enterprise search of the shared database. Once 
identified, the user must assess each risk and assign a risk factor (Rf). For each risk, the 

30 user views the Pf table configured for the current project, selects the category most 
appropriate for the particular risk and selects the qualitative probability definition that 



most closely characterizes the risk thereby specifying a value of Pf. Then the user views 
the Cf table configured for the current project and, on the assumption that the risk occurs, 
selects the cost, schedule and performance impacts based on the anticipated 
consequences. The tool calculates the risk factor Rf as the product of Pf and the highest 

5 Cf of the three and classifies the risk as, for example, low, moderate or high. 

The risks, risk factors and the underlying data are aggregated in the database, 
ranked and provided to a risk review group, which assess the risks at periodic meetings 
and oversees the creation and implementation of mitigation plans. For example, low 
risks may be mitigated by the user but receive no other attention from the board, 

10 moderate risks may warrant a mitigation plan developed and implemented by the user 
who identified the risk with board oversight, and high risks may cause the immediate 
formation of a special review team to develop and manage the mitigation plan. The user 
or board can augment their own knowledge and expertise by conducting a mitigation 
search of the database to identify mitigation plans that have been tried successfully or 

15 unsuccessfully on the same or similar risks in the past. 

As a result, risks are captured, assessed, prioritized and mitigation plans are 
integrated into the Integrated Master Plan before the project starts and the risks actually 
occur. As the plan is implemented it is inevitable that notwithstanding the mitigation 
plans some risks will occur and new risks will arise. The web-based tool facilitates 

20 tracking the status of these risks and generating reports to the management groups so that 
the same management processes can be applied in a timely manner to these risks to keep 
the project on schedule and within budget and to deliver incorporating customer 
specifications. 

These and other features and advantages of the invention will be apparent to those 
25 skilled in the art from the following detailed description of preferred embodiments, taken 
together with the accompanying drawings, in which: 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1, as described above, is a block diagram of a standard risk management 
30 process; 

FIG. 2, as described above, is a typical probability of occurrence (Pf) table for use 

6 



in known risk management tools; 

FIG. 3, as described above, is a typical severity of occurrence table (Cf) for use in 
known risk management tools; 

FIGs. 4a and 4b are block diagrams of a centralized server and a plurality of 
5 interconnected workstations and the web-based risk management tool in accordance with 
the present invention; 

FIG. 5 is a block diagram of a risk management process that embodies the present 
invention; 

FIG. 6 is a web shot of a tailorable probability of occurrence (Pf) table for use in 
1 0 the web-based risk management tool; 

FIG. 7 is a web shot of a tailorable severity of occurrence table (Cf) for use in the 
web-based risk management tool; 

FIGs. 8a and 8b are web shots of an enterprise search form for identifying risks 
from the shared database and exemplary search results; 
15 FIG. 9 is a web shot of a Pf table for a particular project; 

FIG. 10 is a web shot of a Cf table for a particular project; 

FIG. 1 1 is a web shot of the prioritized and aggregated risk factors as presented to 
the risk review board; 

FIGs. 12a and 12b are web shots of a mitigation search for identifying mitigation 
20 plans from the shared database and exemplary search results; 

FIGs. 13a through 13c are, respectively, a screenshot of the Minutes input page 
of the tool, an example of the Minutes output report, and an example of the most 
common risk report (Risk Review Board Report); and 

FIG. 14 is a plot of risk, cost and schedule exposure. 

25 

DETAILED DESCRIPTION OF THE INVENTION 

The present invention provides a risk management tool and method for creating 
an improved initial risk management process that captures, assesses, and prioritizes risks 
and implements mitigation plans to more effectively manage risk in large-scale 
30 development projects. The invention utilizes a web-based system in which users and 
management can share and access risk information to more readily identify and assess 



risks, develop mitigation plans and track the risk management process. The tool and 
specifically the Pf and Cf tables can be tailored to a particular project to more accurately 
calculate and prioritize risk factors RF and thus anticipate and minimize or eliminate 
risks before they occur. The demonstrated value of the tool is found in timely deliveries 
5 and a reduction of cost overruns as well as an increase in customer enhanced award fees. 

As shown in figure 4a, a web-based risk management system 40 for managing 
risk related to a successful completion of a development project comprises a server 42 
having a shared risk database 44 that stores a probability of occurrence (Pf) table 46 and 
a severity of consequence (Cf) table 48, risk identification information 50 and risk 

10 mitigation information 52. A risk management tool 54 located on the server provides 
standardized interfaces for searching, viewing and entering information to and from the 
shared risk data base via a web browser. A computer workstation 56 in communication 
with the server 42 via an intranet 58 is used to enter a Risk Management Plan 59 for the 
current development project and tailor the Pf and Cf tables via a web browser 60. A 

1 5 plurality of computer workstations 62 in communication with the server via the intranet 
are provided for the engineers charged with developing and implementing the risk 
management plan. Each workstation is provided with a web browser 64 to search the 
database 44 to identify risks, to select entries from the tailored Pf and Cf tables to 
calculate and prioritize a risk factor Rf for each risk, to search the database for available 

20 risk mitigation information to develop risk mitigation plans 65 and to track and report on 
the status of identified risks. The actual identified risks, mitigation plans and their 
effectiveness are saved back in the database for access by other workstations on the 
network. 

As shown in figure 4b, the risk management tool 54 is a web-based tool having a 
25 browser interface 66 that allows users to search, view and edit risk information stored on 
the database. The interface 66 includes a menu bar 67 having menu items 68 including 
Login/Logout, Help, Program List, Submit New Risk, Risk List, Reports and Find Risks. 
Clicking through these pull down menu items 68 provides access to hyperlinks 69 that 
allow the user to interact with information on the database, construct searches, define 
30 risks, develop mitigation plans and generate reports and view that information on layout 
pages 73 such as the Login Page, Available Programs Page, Risk List Page, etc. The Pf 



and Cf tables are found from a hyperlink above the pull down menus. 

To support the risk management process shown in figure 5, the web-based tool 54 
is organized and used in part as follows. A user navigates to the Login Page, selects the 
"Login" menu item, enters a name and password and, if validated, enters the Available 
5 Programs Page. The Login Page also includes hyperlinks 69 for New User, Forgot 
Password, Contacts, IPDS and Risk Homepage items. By selecting "New User", any user 
can gain limited access to the tool through a security system that ensures US citizenship. 
There are three levels of access: read-only, write, and administrative. A new user will be 
given read only access to all programs. Write/Administrative access can only be given by 

10 the program creator or by the Database Administrator or Risk Manager. By selecting 
"IPDS", a user can access risk flow charts that describe each of the main steps in the risk 
management process shown in figure 5. 

The Available Programs Page is a list of all the programs using the risk tool that 
have released their information. By clicking on Administration Access, the user can 

15 create a new program or, assuming necessary permissions, can access the Pf and Cf 
tables and tailor them for the current project as detailed in figures 6 and 7. By clicking 
on "Enterprise Risk Search" from the Available Programs Page menu, the user can 
search the shared database to identify risks as detailed in figures 8a and 8b. By clicking 
on a particular "Program Name" in the list, the user can retrieve and view all of the risks 

20 in the current program as detailed in figure 1 1 . Clicking on any risk title within the list 
allows the user to view a Risk Review Board report (figure 13c) that displays all of the 
information about that risk including its mitigation plan and risk assessment Clicking on 
the Edit User Info on the Available Programs Page allows the user to change his/her 
username and password. Clicking on Create New Program allows the user to add a 

25 program to the database. 

The Risk List Page allows the user to Find specific risks from within the program 
list of risks and Sort the risks based upon alphabetical listings, dates or numbers relating 
to all of the fields within the tool. The Find and Sort hyperlinks take the user to the Quick 
Search Page which allows an Advanced Search if the user requires a specific search from 

30 a more detailed list of fields. The Quick Search Page also offers a Mitigation Search 
capability allowing the user to find specific mitigation plans (figure 12b). From the Risk 



List Page, a user with "write" or "administrative" privileges can select "Mit", "Edit", 
"View" and "Hide." The user selects "Mit" to view and edit the current mitigation plan 
for a particular risk as shown in figure 12b. By further selecting "Mitigation Search" the 
user is able to search the database for existing plans or information related to the current 
5 risk as shown in figure 12a. By selecting "Edit" the user is able to edit all of the risk's 
fields except for the mitigation plan. The "View" hyperlink allows the user to see, but 
not edit the entire Risk Review Board Report (figure 13c) on the Risk View Page. Also, 
on the Risk View Page, the user can select a Waterfall Chart for the risk being reviewed. 
The Waterfall Chart is developed in MS Excel based on dates and Pf plus Cf numbers in 

10 the mitigation plan. The Chart graphically displays how the risk is mitigated over time. 
Clicking on "Hide" allows the user to temporarily eliminate a particular risk from the list 
of risks so that a report can be developed omitting that particular risk. 

By selecting "Reports", on the menu bar the user can access a wide variety of 
reports including Risk Quick List, Risk List, Risk Review Board Report, Top 10 Scatter 

1 5 Chart, Top 25 Scatter Chart, Status Report, Assessment Summary, Assessment Rational, 
Mitigation Activity Report, Personnel List, Quad Chart and Minutes which are available 
in Adobe Acrobat PDF, MS Word or MS Excel formats. An example of the interface to 
record minutes of the risk review board and generate the report is illustrated in figures 
13a through 13c. 

20 As illustrated in figure 5, in an initial phase 70 of a risk management process 71 

the program risk manager and a risk review board define a risk management plan 59 
(step 74). The plan is based on customer requirements such as technical specifications, 
cost targets and schedule for the current project and outlines how risks will be identified 
and assessed, how mitigation plans will be developed, how risks will be tracked and what 

25 tools will be used. 

The risk manager and review board use the tool ("Administration Access") to 
tailor the Pf table 46 and Cf table 48 to the current project (step 76) to facilitate 
computation of the risk factor Rf. Experience gained over a number of years and many 
development projects has shown that accurately estimating and prioritizing the risk factor 

30 early enough in the process is key to anticipating and providing mitigation resources to 
the risks rated high enough to cause program disruption. Thus, it is very important to 

10 



create the Pf and Cf tables in such a manner that they are both robust enough to be 
applicable to various programs and a wide range of risks but specific enough to provide 
reliable risk factors. 

As shown in figure 6, the probability of occurrence table 46 maps categories of 
risk 78 to a probability of failure (Pf) 80 based on standardized qualitative probability 
descriptions 82. Currently thirty-two different categories have been defined for 
characterizing different risks at different stages in a development project. These include 
ASSEMBLY, ENGINEERING, EXPECTED LIFE LIMIT, HARDWARE, 
MANUFACTURING, MANAGEMENT, HARDWARE2, MATERIALS, PLATFORM 
INST, PRODUCIBILITY1, PRODUCIBILITY2, PROGRAM, REQUIREMENTS 1, 
SOFTWARE 1, REQUIREMENTS2, SOFTWARE2, SUPPORTABILITY, SUPPORT, 
STRUCTURE, TECH MANUALS, TECHNOLOGY 1 , TECHNOLOGY2, TEST & 
INSPECT, TESTING 1, TESTING2, TRAINING, FREQUENCY, OVERALL, 
RELIABILITY, OBSOLESCENCE, DESIGN HW SW and PROCESS 
REQUIREMENTS. It is important to note that the categories 78 are not the identified 
risks. As a result, it is not necessary for each new project to uniquely define specific risk 
categories and define the probability definition for each probability. This would be 
unwieldy. The probability descriptions 82 are standardized, qualitative and detailed to 
reduce subjectivity of individual engineers and select a more accurate risk factor for the 
current project. The web interface 84 supported by the tool, prompts the manager to 
select from the thirty-two risk categories they want to use in the current project. In 
addition, the interface allows them to define a new category and standardized qualitative 
probability definitions should they so choose. 

As shown in figure 7, the severity of consequence table 48 maps cost impact 86, 
schedule impact 88 and technical/performance 90 to a severity measure (Cf) based on 
qualitative descriptions 91 that are assigned program specific values. In particular, cost 
impact 86 is sub-divided into development cost (NRE) 92, unit cost or design-to-cost 
(DTC) 94 and operations and support cost (O/S) 96 and specified in dollar amounts 98 
for the current project, and schedule impact 88 is specified in actual days, weeks or 
months 100 for the current project. The web interface 102 supported by the tool, 
prompts the manager or review board to select the qualitative descriptions 91 for each 

11 



entry of each impact from a menu of choices or enter a new description. The interface 
also prompts the manager to enter specific dollar amounts for the different cost impacts 
and the number of days, weeks or months for the schedule impact. The dollar amounts 
that correspond to the same severity may be dramatically different for the three cost 
5 impacts. Furthermore, the schedule impact of a delay is tailored to a given project rather 
than being a percentage of the amount of time initially allocated to achieve the milestone. 
The capability to tailor the specific cost impacts and schedule in this manner rather than 
as some fixed percentage is key to forming useful risk factors. It requires more 
consideration and effort up front but it has been demonstrated to be highly effective. 

1 0 Once the risk management plan has been established and the Pf and Cf tables set, 

the risk management process moves into the ongoing cycle 106. Each user identifies 
possible risks associated with his or her assigned task ("Enterprise Risk Search" and 
"Find Risks") (step 108). To augment their own knowledge and experience, an engineer 
will use the web browser 64 to generate an enterprise search 109 of the shared database 

15 as shown in figure 8a. The web interface prompts the engineer to specify a number of 
search parameters 110 including current or historic, risk factor, vendor, component, 
functional area, category, key word in title or description, IPT (Integrated Product Team), 
actionee (engineer), team leader and risk number. The tool searches the shared database 
44 on server 42 and returns possible risks 111 as shown in figure 8b of which the 

20 engineer may accept some, none or all. Each risk has an assigned Risk No, Program, 
Risk Title, and current Risk Factor. By selecting "Transfer" the user can transfer the risk 
including any mitigation plan to his/her program where the user can modify the plan. 
Other tools such as TRIMS, @ Risk, Expert Opinion, Watch Out For List, SEI 
Taxonomy, etc. may be used to identify risks as well. 

25 Once identified, the engineer must enter the risk into the database if the risk was 

not transferred from another program. To enter a risk, the user clicks on "Submit New 
Risk" on the menu bar. A page opens allowing information to be authored for all of the 
risks fields. To assess the risk via the identification of Pf and Cf selections the user 
clicks on the Pf or Cf table hyperlinks. For each risk, the engineer uses the web browser 

30 to view the Pf table 46 configured for the current project as shown in figure 9, selects the 
categories that are 78 most appropriate for the particular risk and selects the qualitative 

12 



probability definitions 82 that most closely characterizes the risk thereby specifying a 
value 80 of Pf. Then the engineers use the web browser to view the Cf table 48 using a 
hyperlink located above the Cf category pull down menu. The Cf table is configured for 
the current project as shown in figure 10 and, on the assumption that the risk occurs, 

5 selects the cost 86, schedule 88 and performance 90 impacts based on the anticipated 
consequences. The cost impact may have three different values corresponding to NRE, 
DTC and O/S. The engineer may formulate a search of the database to access 
information to assist in this selection. For example, the database may store the unit cost 
of an alternate design should the new design fail. Thereafter, the tool calculates the risk 

1 0 factor Rf (step 1 1 2) as the product of the highest Pf and the highest Cf and classifies the 
risk as, for example, low, moderate or high. It is possible to construct Rf as a different 
function of Pf and Cf other than a simple product but this metric has been shown to yield 
the best results. 

The risks, risk factors and the underlying Pf and Cf data are aggregated in the 

1 5 shared database 44, ranked and provided to a risk review board, which review the risks at 
periodic meetings and oversees the creation, funding and implementation of mitigation 
plans (step 114). As shown in figure 1 1, the risk list 113 ("Risk List") ranks the risks 
from highest to lowest risk factor and provides the Risk #, Title and Actionee for each 
risk. The web interface allows the user to "Edit" the risk description, actionee, 

20 assessment, etc. , "Mit" to view and edit the risk mitigation plan, "View" the Risk Review 
Board Report, and "Hide" to hide a risk. Based on this information, for example, low 
risks may be placed on a watch list and left to the actionee (engineer) to mitigate and 
receive no other attention from the board; moderate risks may warrant a mitigation plan 
developed and implemented by the identifying engineer and overseen by the board; and 

25 high risks may cause the immediate formation of a special review team including 
possibly the customer to develop and manage the mitigation plan. Depending upon the 
size of the project, this process may occur at multiple levels and be subjected to 
overlapping mitigation plans. 

Risk review board meetings are typically held monthly. The customer involved 

30 meeting is held to discuss high, moderate and new risks. During the monthly meeting, 
the proposed risk-handling or mitigation plan is discussed and resources are authorized 

13 



and applied. To develop the risk mitigation plan, the risk actionee creates the plan with 
the aid of the risk management tool. The tool allows the actionee to research current 
mitigation plans as well as past successfully implemented mitigation plans. As shown in 
figure 12a, a mitigation search 116 ("Mitigation Search 55 ) of the database includes a 

5 description of the risk and current status, a start date, original planned complete date, an 
expected planned complete date and actual complete date. From this the actionee is able 
to share expenses with other programs with the same or similar mitigation plans, 
determine pitfalls of past mitigation plans, and develop a plan based upon successful 
plans. Of course, if the original assessment of Rf was not accurate, resources may be 

1 0 wasted mitigating over prioritized risks or not assigned to address under prioritized risks. 
This is the reason that precise tailoring of the Pf and Cf tables from the beginning of the 
process is done. 

As a result, risks are captured, assessed, prioritized and mitigation plans 65 are 
integrated into the Integrated Master Plan before the project starts and the risks actually 

15 occur. As shown in figure 12b, a mitigation plan 65 for a single risk may have multiple 
mitigation activities. Each activity is numbered, provided with a description, a Pf, a Cf, 
planned completion date, actual completion date and a description of the results. This 
window is accessed through "Mit" and is used for editing purposes. At this point, the 
initial risk mitigation plans 65 are provided to a program schedule manager and schedule 

20 review board to incorporate into the master plan and schedule. Although risk 
management is an important aspect of scheduling, they are generally treated as separate 
functions and the details of scheduling will not be addressed. To simplify, a tool such as 
Microsoft Project is used to create a detailed schedule that links all the tasks and sub- 
tasks and mitigation plans together. As the project progresses, the engineers and 

25 managers continually update MS Project to track achieved milestones, slippage and 
reasons for revision. 

As the schedule is implemented and the project progresses it is inevitable that 
notwithstanding the mitigation plans some risks will occur and new risks will arise. The 
progress information from MS Project is forwarded to the risk manager to track and 

30 report risks (step 118). The web-based tool streamlines the process of preparing and 
submitting status reports. As shown in figures 13a through 13c, the web-tool provides a 

14 



template 120 for inputting minutes 121 from the periodic risk review board meetings to 
track each risk and generate a report 122 (step 124) that is maintained on the shared 
database 44. The risk review board (RRB) minutes 121 are generated by clicking "Input 
New RRB info" on template 120 and submitting general information from the meeting. 

5 The RRB date and the risks covered during the meeting are required fields. Minutes for 
each risk are submitted by clicking "Input/Update Minutes for Risks Covered". Once all 
the minute information is submitted, the user returns to "RRB Minutes Home", selects 
the meeting date under "View RRB report" and clicks the "Create Minutes Report" to 
generate the report 122. The report includes Number, Title, Actionee, Rf, Risk Level and 

10 Comments for each risk inserted. This automated creation of minutes allows the user, 
program manager and board to quickly review and manage many complicated risks in a 
large-scale development project. 

As risks are successfully mitigated and the milestones achieved, the risks are 
closed (step 126). If a risk changes, the engineer/board reassesses the probability of 

1 5 occurrence and impact (step 112), makes required adjustments to the mitigation plan and 
submits a program plan adjustment report (step 128). In some cases, a Monte Carlo 
simulation such as Risk Plus is run on the schedule based on the most optimistic and 
pessimistic milestone dates and generates additional scheduling risks that are considered. 
If a new risk or task appears, the engineer identifies the risks (step 108) and repeats the 

20 process. If the scope of the project changes, the process returns to the initial phase for 
reconsideration of the risk management plan (step 74). 

As part of tracking and reporting risks, the tool automatically captures and 
displays valuable information to measure process implementation. Most of the 
information is automatically captured and/or calculated. Numbers of risks being 

25 mitigated or reduced in severity, numbers of risks being identified, and risk exposures are 
automatically determined monthly for every program. The risk exposure is defined as the 
sum of the cost multiplied times the probability of each risk for the entire program. This 
number has been found to be an accurate predictor of cost overruns when proper 
mitigation is not implemented. Without accurate capture of dollar impacts for each risk 

30 provided by the tailored Cf table within the tool, this calculation would not be possible. 
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Figure 14 illustrates how the cost exposure 130 of a program increases as risks are 
identified and slowly decreases as soon as risks begin to be mitigated. At the same time 
the identification of risks 132 continues to go up showing that the overall risk process is 
successfully being implemented. If this cost exposure table were to show the cost 
5 exposure continuing to increase, then the program would be doing well in the 
identification of risk, but not necessarily the mitigation of risks. Note that in this 
situation, risks were identified and properly mitigated and continue to be identified 
throughout the life of the program. If the risk process is implemented properly, the risk 
schedule exposure 134 will follow the trend of the cost exposure 130. Schedule exposure 

10 is defined using the same calculation, but is not used as a predictor of schedule overrun. 
It is used as a measure of schedule overrun trends. 

The following detailed example is provided to illustrate the effectiveness of the 
risk management tool of the present invention and to emphasize the importance of a web- 
based tool that facilitates sharing of information between engineers and projects and the 

15 importance of carefully tailoring the probability of occurrence Pf and severity of 
consequence Cf tables for each program. 

Two engineers, engineer A and engineer B, are working as team leads for 
different components of a widget. Engineer A has been an employee for a long time. 
Engineer B has much less experience. The following two scenarios, Scenario S 

20 (Standard Process) and Scenario R (New Process), tell the story of engineers A and B 
going through the risk process using the Standard Risk Process and tools described with 
reference to figures 1 through 3 above (Scenario S) and the Risk Management Process 
and Web based Tool of the present invention (Scenario R). In both scenarios the process 
begins before Engineer A and B enters the story. A Risk Management Plan is developed 

25 for the Widget program by the program management. This is where the two scenarios 
diverge. 

Scenario S: 

Risk Management Plan: The Risk Management Plan is developed (Step 14). 
30 Within the Plan it is defined that the widget program engineers will use their expert 
opinions to identify risk (Step 16). Once risks are identified, the Plan dictates the 
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engineers shall use the Probability of Occurrence Table, shown in figure 2, to select a 
probability value for each risk. The engineer uses the Severity of Consequence table, 
shown in figure 3, to select a severity value for each risk. The probability value is 
multiplied by the severity value for each risk to determine a risk factor. The risks are 
5 ranked by this risk factor (Step 18). The engineers develop mitigation plans for their 
risks, using their expertise (Step 24). The Risks and Mitigation plans are entered into the 
Standard Risk Tool that stores the risks and creates reports for the widget program. The 
status of the mitigation plans and risks are tracked and reported throughout the life of the 
program. 

10 Risk Identification: Engineer A (widget battery team lead), using his vast 

experience, realizes there will be risk with the design of the battery. It is a brand new 
design that has never been implemented before, though some analysis has shown it 
feasible. Engineer A identifies this as a risk, following the Risk Management Plan. At 
the same time Engineer B (widget housing team lead), being a less experienced engineer, 

15 is worried there will be risk redesigning the old housing of a similar widget. Engineer B 
identifies this as a risk, following the Risk Management Plan. Additionally, Engineer B 
does not realize that the vendor supplying the material for the housing has issues with 
delivering on time in the past, and thus no risk for this is identified. 

Risk Assessment and Prioritization: Engineer A, following the Risk Management 

20 Plan, uses the Probability of Occurrence Table (Figure 2) and the Severity of Occurrence 
Table (Figure 3) to assess his risk. He assesses the probability as a point .3, determining 
the probability of the risk occurring is not "likely", even though he wouldn't describe it 
as "unlikely" either, he believes it is closer to this definition. Next he assigns the 
Consequence of his risk. If he is not able to sufficiently design the new battery, an old 

25 battery will have to be used, which will cost $ 1 00 more than the current battery for each 
unit (Recurring Cost) and $500,000 in redesign costs (Non Recurring Cost) to make the 
old battery fit into the widget design. Ordering the old battery would also add 3 months 
to his delivery schedule. Not sure which cost increase to use for the cost severity he 
chooses the unit cost, thinking that is the programs priority, and knowing the increase is 

30 about 5% of the current battery he assesses a severity of cost of 2. He also notes the 
schedule severity, but can not enter both cost and schedule so the severity stays at 2. 
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When the probability, .3, is multiplied by the severity, 2, the risk factor is a .6. 

At the same time engineer B assesses his risk for the redesigned housing. Using 
the probability table he assesses the probability to be a .7, thinking it is "highly likely" at 
least something will go wrong with the design. Next he assessed the severity. Knowing 

5 the housing must be redesigned to fit the widget, he knows it will take extra money and 
time to get the design right should the design not work the first time. He assesses the 
severity of schedule as a 5, thinking that slipping his delivery would be "can't achieve 
key milestone", since the milestone is key to him. When the probability, .7, is multiplied 
by the severity, 5, the risk factor is 3.5. 

10 Program management next prioritizes the program risks by risk factor. Because 

the engineers assessed the risks subjectively based on the vague tables, the housing risk 
shows up much higher than the battery risk, based on the fact Engineer A had to leave out 
the schedule impact of the battery risk and both engineers were greatly influenced by 
their own perceptions and personalities when assessing the probability and severity. 

1 5 Mitigation: Since the housing risk is one of the highest, the program management 

helps Engineer B with the mitigation plan, by assigning two more people to the housing 
team to help with the design. Meanwhile Engineer A, comes up with a mitigation plan 
on his own that includes some extra testing during the design to catch problems early on. 
He does not know that a battery tester was developed on another program to mitigate a 

20 similar risk, so he spends resources creating a battery tester from scratch. 

Tracking and Reporting: Both risks are entered into the standard risk tool and are 
tracked by program management. 

Summary: Since Engineer B missed identifying the risk with the housing material 
vendor, nothing was done to mitigate their poor delivery history. Characteristically, the 

25 vendor delivered two months late. Engineer A spends an additional $250,000 to develop 
an engineering battery tester, not knowing one already existed on another program. 
Engineer B has more engineers than he needs for his redesign of an old housing and 
Engineer A has difficulties with the brand new design, causing additional cost and 
schedule delays. After delays and costs are incurred, program management transfers 

30 designers from engineer B's team over to engineer A's team. 
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Scenario R: 

Risk Management Plan: The Risk Management Plan is developed (step 74). The 
Plan dictates that the widget program engineers use their expert opinions and risks from 
other company programs using the shared database (shown in figure 4) to identify risk 
(step 108). Once risks are identified, the Plan dictates the engineers use the Probability 
of Occurrence Table, shown in figure 9, with categories chosen specific to the program, 
to select a probability value for each risk. The engineer then uses the Severity of 
Consequence table, shown in figure 10, with actual cost and schedule values chosen by 
the program, to select a severity value for each risk. The probability value is then 
multiplied by the severity value for each risk to determine a risk factor. The risks are 
ranked by this risk factor (step 112). The engineers develop mitigation plans for their 
risks, using their expertise and mitigation plans created for similar risks on other 
programs using the shared database (step 114). The Risks and Mitigation plans are 
entered into the Web based Risk Tool that stores the risks and creates reports for the 
widget program. The status of the mitigation plans and risks are tracked and reported 
throughout the life of the program. 

Risk Identification: Engineer A (widget battery team lead), using his vast 
experience, realizes there will be risk with the design of the battery. It is a brand new 
design that has never been implemented before, though some analysis has shown it 
feasible. He also checks the Web based shared Risk database for similar battery risks. 
He finds a few but none are applicable to what he is doing. Engineer A identifies this as 
a risk, following the Risk Management Plan. At the same time Engineer B (widget 
housing team lead), being a less experienced engineer, is worried there will be risk 
redesigning the old housing of a similar widget. Engineer B identifies this as a risk, 
following the Risk Management Plan. Additionally, Engineer B checks the Web based 
shared Risk database for similar related risks. While searching on the vendor that will 
supply the material, he finds a risk from another program that details the vendor's 
previous failures to meet delivery dates. He also identifies a risk for this and will use the 
same mitigation steps used by the other program to mitigate the risk. 

Risk Assessment and Prioritization: Engineer A, following the Risk Management 
Plan, uses the Probability of Occurrence Table (figure 9) and the Severity of Occurrence 
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Table (figure 10) to assess his risk. He assesses the probability as a point .8, based on the 
Hardware category and the definition "New design supported by some analysis". Next he 
assigns the Consequence of his risk. If he is not able to sufficiently design the new 
battery, an old battery will have to be used, which will cost $100 more than the current 
5 battery for each unit (Recurring Cost) and $500,000 in redesign costs (Non Recurring 
Cost) to make the old battery fit into the widget design. Ordering the old battery would 
also add 3 months to his delivery schedule. He assesses a severity for Unit cost of .2, for 
NRE cost of .5, and a schedule cost of .7. He is allowed to enter a value for each unit 
cost, NRE cost, and schedule. When the Web tool calculates the Risk Factor, it will take 
10 the highest value of the three (.7) to multiply by the probability. When the probability, 
.8, is multiplied by the severity, .7, the risk factor is a .56 and is assigned a risk rating of 
High. 

At the same time engineer B assesses his risk for the redesigned housing. Using 
the probability table he assesses the probability to be a .5, based on the Hardware criteria 

1 5 of "major redesign of existing components or major functional mods". Next he assesses 
the severity. Knowing the housing must be redesigned to fit the widget, he knows it will 
take extra money and time to get the design right should the design not work the first 
time. He determines worst case it would take 2 months extra and cost $300,000 in extra 
NRE. Though this seems severe to him, after looking at the severity table, he assesses 

20 the severity of schedule as a .5, and the NRE cost as a .3. When the probability, .5, is 
multiplied by the highest severity, .5, the risk factor is a .25 and is assigned a risk rating 
of Moderate. 

Program management next prioritizes the program risks by risk factor. Because 
the engineers assessed the risks objectively using the program tailored assessment tables, 
25 the battery risk shows up higher than the housing risk, based on the fact it is a brand new 
design and has greater cost and schedule consequences. 

Mitigation: While developing a Mitigation plan for the battery risk, Engineer A 
searches Mitigation plans on the Web based shared database for similar risks. He 
discovers the battery tester developed on another program and is able to borrow the tester 
30 from them. Management also adds additional engineers to his team to help with the 
design. Engineer B develops a mitigation plan for his risk using his expertise and some 
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similar examples found on the Web based shared database to develop a mitigation plan 
for his risks. 

Tracking and Reporting: Both risks are entered into the Web based shared 
database where they can be viewed by all program personnel and are shared with 
personnel from other programs. 

Summary: Since engineer B identified the housing material vendor risk, a special 
team was sent to the vendor to ensure on time delivery. The housing material was 
delivered a week early. Since engineer A had discovered the battery tester from the other 
program, he was able to borrow it and save the cost of developing his own. With his risk 
being the highest, extra designers were assigned to his team allowing his design to 
complete on time. Despite his initial worries, engineer B's redesign of the old housing 
was simpler than expected and completed successfully within time and budget. Thanks 
to the correct risks being identified, and the risks being assessed objectively on the same 
scales, the risks were mitigated adequately for the program to finish on time and on 
budget. 

While several illustrative embodiments of the invention have been shown and 
described, numerous variations and alternate embodiments will occur to those skilled in 
the art. Such variations and alternate embodiments are contemplated, and can be made 
without departing from the spirit and scope of the invention as defined in the appended 
claims. 
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